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Abstract. Using Galois cohomology, Schmoyer characterizes crypto- 
graphic non-trivial self-pairings of the ^-Tate pairing in terms of the 
action of the Frobenius on the ^-torsion of the Jacobian of a genus 2 
curve. We apply similar techniques to study the non-degeneracy of the 
^-Tate pairing restrained to subgroups of the ^-torsion which are max- 
imal isotropic with respect to the Weil pairing. First, we deduce a cri- 
terion to verify whether the jacobian of a genus 2 curve has maximal 
endomorphism ring. Secondly, we derive a method to construct horizon- 
tal (^, ^)-isogenies starting from a jacobian with maximal endomorphism 
ring. 



1 Introduction 

A central problem in elliptic and hyperelliptic curve cryptography is that of 
constructing an elliptic curve or an abelian surface having a given number of 
points on their Jacobian. The solution to this problem relies on the computation 
of the Hilbert class polynomial for a quadratic imaginary field in the genus one 
case. The analogous genus 2 case needs the Igusa class polynomials for quartic 
CM fields. There are three different methods to compute these polynomials: 
an analytic algorithm [TJ], a p-adic algorithm [B] and a Chinese Remainder 
Theorem-based algorithm The last one relies heavily on an algorithm for 
determining endomorphism rings of the jacobians of genus 2 curves over prime 
fields. 

Eisentrager and Lauter [4J gave the first algorithm for computing endomor- 
phism rings of Jacobians of genus 2 curves over finite fields. The algorithm tales 
as input a jacobian J over a finite field and a primitive quartic CM field K, 
i.e. a purely imaginary quadratic extension field of a real quadratic field with 
no proper imaginary quadratic fields. The real quadratic subfield Kq has class 
number 1. It computes a set of generators of an order O in the CM field and tests 
whether these generators are endomorphisms of J, in order to decide whether 
the order O is the endomorphism ring End( J) or not. In view of application to 
the CRT method for computing Igusa class polynomials, Freeman and Lauter 
bring a series of improvements to this algorithm, in the particular case where we 
need to decide whether End(J) is the maximal order or not. 



Note that the Eisentriiger-Lauter CRT method for computing class polyno- 
mials searches for curves defined over some prime field ¥p and belonging to a 
certain isogeny class. Once such a curve is found, the algorithm keeps the curve 
only if it has maximal endomorphism ring. This search is rather expensive and 
ends only when all curves having maximal endomorphism ring were found. Re- 
cent research in the area j lll4l3| has shown that we can significantly reduce the 
time of this search by using horizontal isogenies, i.e. isogenics between jacobians 
having the same endomorphism ring. Indeed, once a Jacobian with maximal en- 
domorphism ring is found, many others can be generated from it by computing 
horizontal isogenies. 

In this paper, we propose a new method for checking if the endomorphism 
ring is locally maximal at I, for i > 2 prime. Our method relies on the com- 
putation of the Tate pairing. We study subgroups of the i'-power torsion which 
are maximal isotropic with respect to the Weil pairing and such that the Tate 
pairing restricted to these subgroups is fc^.j-degenerate (in the sense of Defi- 
nition [T]). We show that the computation of fc^.j suffices to check whether the 
endomorphism ring is locally maximal at £, in many cases. Moreover, we give a 
method to distinguish kernels of horizontal {£, ^)-isogenies from other isogenies of 
principally polarized abelian varieties. Our main result is the following theorem. 

Theorem 1. Let H be a hyperelliptic curve defined over a finite field ¥q and 
£ > 2 a prime number. Let J be the jacobian of H , whose endomorphism ring is 
a locally maximal order at £ of a CM- field K . Suppose that the Frobenius endo- 
morphism is exactly divisible by £" , n e Z and that the conditions in Lemma\^ 
are satisfied. Then a subgroup G C J[£], which is maximal isotropic with respect 
to the Weil pairing, is the kernel of a descending isogeny if the Tate pairing 
is ki^j -non-degenerate over G x G, for G C J[£"] such that £"^^G — G and 
that G is maximal isotropic with respect to the £"'-Weil pairing. The isogeny is 
horizontal if the Tate pairing is kij -degenerate over G x G. 

In view of application to the CRT method for Igusa polynomial computation, we 
deduce an algorithm to compute kernels of horizontal isogenies efficiently. This 
generalizes a result on horizontal ^-isogenics for genus 1 curves [8]. 

This paper is organised as follows. In Section[2]we recall briefly the Eisentrager- 
Lauter algorithm for computing endomorphism rings. In Section [3] we give the 
definition and properties of the Tate pairing. Section U] describes our algorithm 
for checking whether a Jacobian has locally maximal order at £. In Section [S] we 
show that we can compute kernels of horizontal {£, ^)-isogenies by some Tate pair- 
ing calculations. Finally, Section [6] gives complexity estimates for our algorithms 
and compares their performance to that of the Freeman-Lauter algorithm. 

Notation and assumptions. In this paper, we assume that principally polarized 
abelian surfaces are simple, i.e. not isogenous to a product of elliptic curves. A 
quartic CM field if is a totally imaginary quadratic extension of a totally real 
field. We denote by Kq the real quadratic subfield of K and we assume that Kq 
has class number 1. We assume that K — Q{r]), with ij = i\J a -\- bVd if d = 2,3 



mod 4 or rj = a + b y ^2 ) d. = I mod 4. A CM-type <P is a couple of 
pairwise non-complex conjugate embeddings of K in C 

An abelian surface over C with complex multiplication by Ok is given by A{C) = 
C^/<?(a~^), where a is an ideal of Ok and <P is a CM type. This variety is said 
to be of CM-type {K, A CM-type {K, ^) is primitive is (p cannot be obtained 
as a lift of a CM-type of a CM-subfield of K. The principally polarized abelian 
variety C'^ /<P{a~^) is simple if and only if its CM-type is primitive [13J. 



2 Computing the endomorphism ring of a jacobian 

The endomorphism ring of an ordinary jacobian J over a finite field ¥q {q = p") 
is an order in a quartic CM field K such that 

Z[7r,7f] C End(J) C Ok, 

where Z[7r,7f] denotes the order generated by tt, the Frobenius endomorphism 
and by tt, the Verschiebung. We give a brief description of the Eisentrager-Lauter 
algorithm [4J which computes the endomorphism ring of J. For a fixed order O 
in the lattice of orders of K, the algorithm tests whether this order is contained 
in End(J). This is done by computing a Z-basis for the order and checking 
whether the elements of this basis are endomorphisms of J or not. In order to 
test if a S O is an endomorphism, we write 

a + bn + CTT^ + d-K"^ 

a = 

n 

with a, 6, c, d, n some integers such that a, 6, c, d have no commun factor with n 
{n is the smallest integer such that na S Z[7r]). The LLL algorithm computes 
a sequence a,b,c,d,n such that a can be written as in Equation [TJ In order to 
check whether a is an endomorphism or not, Eisentrager and Lauter [4] use the 
following result. 

Lemma 1. Let A be an abelian variety defined over a field k and n an integer 
coprime to the characteristic of k. Let a : A ^ A be an endomorphism of A. 
Then A[n\ C Ker a if and only if there is another endomorphism ji of A such 
that a — n ■ j3. 

Using Lemma [TJ we get a S End (J) if and only if a + bn -\- ctt^ -f dn^ acts 
as zero on the n-torsion. Freeman and Lauter show that n divides the index 
[Ok ■ ^[tt]] (see [SJ Lemma 3.3]). Since Z[7r,7f] is 1 or p, we have that n divides 
[Ok ■ Z[7r, 7f]] if {n,p) = 1. Moreover, Freeman and Lauter show that if n factors 
as if^ii^ ...If-, it suffices to check if 

a + b-K + c-K^ + d-K^ 



(1) 



for every prime factor in the factorization of n. The advantage of using this 
family of elements instead of a is that instead of working over the extension field 
generated by the coordinates of the n-torsion points, we may work over the field 
of definition of the -torsion, for every prime factor li. For a fixed prime 
Freeman and Lauter prove the following result, which allows computing a bound 
for the degree of the smallest extension field over which the ^-torsion points are 
defined. 

Proposition 1. JS, Prop. 6.2] Let J he the Jacobian of a genus 2 curve over 
¥q and suppose that End{J) is isomorphic to the ring of integers Ok of the 
primitive quartic CM field K. Let i q be a prime number, and suppose ¥pr is 
the smallest field over which the points of J[£] are defined. If £ is unramified in 
K , then r divides one of the following: 

(a) i ~ 1, if i splits completely in K; 

(b) l"^ ~ \, if £ splits into two or three ideals in K; 

(c) £^ -£^ +£-l, if£is inert m K. 

If i ramifies in K , then r divides one of the following: 

(a) £^ — £'^ , if there is a prime over i of ramification degree 3, or if £ is totally 
ramified in K and £> i. 

(b) £'^^£, in all other cases where £ factors into four prime ideals in K (counting 
multiplicities) . 

(c) £^—£, if £ factors into two or three prime ideals in K (counting multiplicities). 

Once we computed the extension field over which the £-torsion is defined, the 
f'^-torsion will be computed using the following result [2] . 

Proposition 2. /5, Prop. 6.3] Let A he an ordinary abelian variety defined over 
a finite field ¥q and let £ be a prime number not equal to the characteristic of 
¥q. Let d be a positive integer. If the £-torsion points of A are defined over ¥q, 
then the £'^ -torsion points are defined over F^gd-i . 

3 Background on the Tate pairing 

Consider now H a hyperelliptic genus 2 curve defined over a finite field Fg, with 
q — p^, whose equation is 

y^ + hix)y^fix), (2) 

with h, f G ¥q[x], deg h < 2, f monic and deg / < 4. Let J be the jacobian of H 
and denote by the algebraic closure of F^ and by Gf^/f^ — Gal{Wq/¥q) the 
Galois group. Let m G N and consider J[m] the subgroup of m-torsion, i.e. the 
points of order m. We denote by C Fg the group of m-th roots of unity. The 
m-Weil pairing 

Wm : J[m] X J[m] fj,m 



is a bilinear, non-degenerate map and it commutes with the action of G. If 
A : A — > A is a principal polarization, then we define the Weil pairing as 

Wm : J[m] X J[m] fi^ 

{P,Q)^W,n{P-MQ))- 

Given a subgroup G C J[m], we say that G is isotropic with respect to the Weil 
pairing if the Weil pairing restricted to G x G is trivial. It is maximal isotropic 
if it is isotropic and it is not properly contained in any other such subgroup. We 
denote by H^{Gf^/f^, J) the i-th Galois cohoniology group, for i > 0. 

Consider the exact sequence J[m] — > J(Fg) — t- J(Fq) 0. Then by 
taking Galois cohomology we get the connecting morphism 

6 : J(F,)/mJ(F,) = H°{G^jr,,J)/rnH°{Gfjw,,J) ^ H\Gfjr,,J[m]) 

where the map Fp is defined as follows 

Fp:G^jf^^J{¥,)[m] 
a a{P) - P, 

where P is any point such that mP = P. Using the connecting morphism and 
the Weil pairing, we define the m-Tate pairing as follows 

J(F,)/toJ(F,) X J[m](F,) ^ H'{G,firn) 

{P,Q)^[a^W„r{Fp{a),Q)]. 

For a fixed principal polarization X : J ^ J we define a pairing on J itself 

tii; .) : J(F,)/mJ(F,) x J[m]{¥,) ^ FJ/F^™ 

{P,Q)^t^{P,X{Q)). 

Most often, if J has a distinguished principal polarization and there is no risk 
of confusion, we write simply tm{', •) instead of t^{-, ■). 

Lichtenbaum |10| describes a version of the Tate pairing on Jacobian varieties. 
More precisely, suppose we have m\=f^J(¥q) and denote by k the embedding degree 
with respect to m, i.e. the smallest integer fc > such that m\q'' — 1. Let 
Di G J(Fqfc) and D2 £ J[m]{¥qk) two divisor classes, and let Di be represented 
by Di and D2 by D2 such that supp(£'i) n supp(Z32) = 0- Since D2 has order 
m, there is a function fm,D2 is such that div{fjn,D2) — 'mD2- The Tate pairing 
of the divisor classes Di and D2 is computed as 

trniDi,D2) = fDADl). 

Moreover, in computational applications, it is convenient to work with a unique 
value of the pairing. Given that ¥*k/{¥gk)™ ~ /x^, we use the reduced Tate 
pairing, given by 

T„(-, •) : J{¥gk)/mJ{¥g.) x J[m](F,.) ^ /i„ 



The function /m,D2(^i) is computed using Miller's algorithm [TT] in O(logTO) 
operations in ¥„k. Since H^{Gf — fJ-m by Hilbert's 90 theorem, it fol- 

lows that there is an isomorphism (Gw /p F, ^m) ~ (Gal(F„fcm /F„fc ), /im)- 

Since H^{Gal{¥gkm /¥qk), ^m) — A*™, we may compute the Tate pairing as 

t,n{-, •) : J{¥^k)/mJ{¥qk) x J[m](F,0 ^ /i™ 

(F,Q)^W^™(Fp(7r),g), 

where tt is the Frobenius of the finite field F^i- . 

4 Pairings and endomorphism ring computation 

In this section we relate some properties of the Tate pairing to the isomorphism 
class of the endomorphism ring of the Jacobian. Let £ be a prime odd number. 
We give a method to check whether the endomorphism ring is locally maximal 
at £ (i.e. the index [Ok ■ O] is not divisible by £) by computing a certain number 
of pairings. 

Let H he a. genus 2 curve defined over a finite field Fg, J its jacobian and 
suppose that J[£"] C J{¥g) and that J[£"+'^] ^ J(F,), with £ different from p 
and n > 1. We denote by W the set of maximal isotropic subgroups in J[£"] 
with respect to the £-Weil pairing and we define ki^j to be 

fc^.j = max{fc|3P, Q G G and Tf7i{P,Q) £ 

Definition 1. Let G be a rank 2 subgroup o/ J[£"] in W . We say that the Tate 
pairing is ki^j -non-degenerate (or simply non-degenerate) onGxG if its restric- 
tion 

Tin : G X G — )■ iJ-fkf j 

is surjective. Otherwise, we say that the Tate pairing is ki^j -degenerate (or simply 
degenerate) on G xG. Moreover, for two divisor classes Di, D2 € G, we say that 
they have non- degenerate pairing if Tgn^Di, D2) is a £^'^--' -th root of unity and 
degenerate otherwise. 

Lemma 2. The reduced Tate pairing defined as 

Tir. : J[e'] X J[r] fie^ 
is ki^, J -antisymmetric, i.e. T^n^Di, D2)Tin{D2, Di) g l^-^ke,.! , for all Di,D2 € 

Proof Indeed, assume that there_are_Z)i , 1)2 € J[£"] such that T^^ {Di,D2)Tin. {D2,Di) G 
jJ't^XlJ'fkf j . We denote by G = {Di, D2) and by r > ki^j the largest integer such 
that Tin[Di, D2)Tin{D2, Di) is an ^''-th primitive root of unity. Then the poly- 
nomial 



V{a, b) = log Ti. {Di,Di)a^ + log(T,. {Di , D2)T,^ {D2,D,))ab + log T,. {D2, D2)b^, 



where the log function is computed with respect to some fixed £"-th root of 
unity, is zero mod £"■-''-1 and non-zero mod Dividing by we may 

view P as a polynomial in ¥([a, b]. Since P is a quadratic non-zero polynomial, 
it has at most two roots. These correspond to two divisor classes in G, with 
r-degenerate self-pairing. Hence, there is at least one divisor D € G such that 
T£n[D, D) is a £'^-th root of unity. Since there is at least one maximal isotropic 
subgroup W e W with respect to the Weil pairing such that D G W, this 
contradicts the definition of k^^j. 

Let O be an order of K and let G O. We define 

ve,o{0) ■■= max{TO : 6* G Z £""0}. 

' m>0 

We denote by 1, 5, 7, r/ a Z-basis of O and and we write 6 = ai+ + 037 + 047/. 
Then we compute ve^o as 

ve,o{6) = ve{gcd{a2,a3,a4)). (3) 

Note that the value of vg^o (^) is independent of the choice of the basis. We say 
that is divisible by i € Z if we have 6 G tO. We say that 9 is exactly divisible 
by £" if it is divisible by ^" and it is not divisible by The following lemma 

gives a criterion to check whether an order is locally maximal at i or not. 

Lemma 3. Let K := Q{i^/ a + h\fd) he a quartic CM field, with r} = i\J a + b- ^~^^ , 

if d = 1 mod 4 and rj = i\/ a + b\fd, if d = 2, 3 mod 4. We assume that a, 6, d G Z 
and that d and — b^d are square free. Assume that Kq = Q(-\/d) has class 
number 1. Let £ > 2 a prime number that does not divide lcm{a,b,d). Let 
Ok be the maximal order of K and O an order such that [Ok '■ O] is di- 
visible by £. Let n G O such that Nk/Ko{''^) & is not divisible by £ and 
that vi^OkM > 0- ^6 suppose that n = ai + 02 + (as + a-i ~"^^^ )?7, 

if d = 1 mod 4 and tt = oi + a2\/rf + (03 -I- a4^/d)ri, if d = 2,3 mod 4. If 
ve{a3 - 04) < mm{ve{a3),ve{a4)), then ve,o{T^) < v^OkM- 

Proof. We denote by Oi = Okq + ^KoV- Since € > 2, it suffices to show that 
VonOi (^) < (■"')• We will therefore assume, without restricting the generality, 
that O C Oi. Let 6 = if d = 1 mod 4 and 6 = Vd, ii d = 2,3 mod 4 and 

let 7 := St]. Then 1, S, 7, 77 is a basis for Oi. We write n = ai + + 037 + 047/. 
By writing down the norm condition for d = 2, 3 mod 4 



oi + 02 



Vd + (03 + a4^\fd)i\J a + bVd^ ^ai -|- a2^/d — (03 + ai\fd)i\J a + bVd^ G Z, 



we get that 



2aia2 + a^b + a^bd + 200304 = 0. 



(4) 



Similarly, for d = 1 mod 4, we have 



ai aai aib ai(l + d)b ^ 

-y + aia2-^+aa3Q4 + ^+ ^ = 0- (5) 

Since £ \ ai, equations ^ and Q imply that W£(a2) > max(wf (03), (04)). 
Since there is always an order O' such that O C C C Oi such that [Ox : 
O'] is a power of £, it suffices to prove the lemma in the case [Oi : O] is a 
power of £. For the order O, we choose {1, S', 7', t]'} a HNF basis with respect to 
{1, 5, 7, 77}. We denote by (fli j )i<i j<4 the corresponding transformation matrix. 
Then [Ok ■ O] — ni<i<4 Note that neither 77 nor 7 are in O. Otherwise, O 
is the maximal order. Indeed, assume rj £ O. Since £ divides neither a nor 6, it 
follows that (5 e C This implies that O is the maximal order. We consider the 
decomposition of tt over the basis {1, 5', 7', r/} 

'1 ' JC' I ' ' I ' ' ' ^ u 
TT = flj^ + fljO + 037 + 0477 , flj G 

Since -q ^ O , we know that 044 is £. If 033 is divisible by then vi{a'^) < velas). 
If 034 = 1, then 04 = —(03 — 04)/^. If 034 = 0, then a'^ = 04/^?. If 033 = 1, 
it follows that 034 = 1 (otherwise we would have 7 e O). Then — 03 and 
04 = —(03 — a4)/£. We conclude that v^ Q{7r) < vi qj^{'k). 

Since we know that J[£^] is Fg-rational, while J[£""'"^] is not, Lemma[l]implies 
that TT — 1 is exactly divisible by Moreover, the Frobenius matrix on the Tate 
module is the identity matrix Z^modf". The following lemma computes the 
precision up to which the Frobenius matrix on the Tate module is of the form 
A/4, with A G Z. 

Lemma 4. Let J be a abelian surface defined over a finite field Fg and tt the 
Frobenius endomorphism. Then the largest integer m such that the matrix of the 
Frobenius endomorphism on the £-Tate module is of the form 



A 
A 
yo A/ 



mod £"" (6) 



is V(_q{t:), where O is the endomorphism ring of J . 

Proof. Let m be the largest integer such that the matrix of the Frobenius on 
J[£"'] has the form given in Equation ([6]). Let O be the endomorphism ring of J. 
We denote by {1, (5, 7, 77} the Z-basis of O and by tt = ai + 028 + 037 + 047? the 
decomposition of tt over this basis. It is obvious that m > ve(gcd{a2, 03, 04))- For 
the converse, we note that tt — A kills the ^'"-torsion, hence we may write vr — A = 
with a E End(J). We write down the decomposition of a over the basis 
{1,(5,7,7;} and conclude that gcd (02, 03, 04). Hence m < U£(gcd(a2, 03, 04)). 
We conclude that m — w^(gcd(a2, 03, 04)), hence 777- = vg^oi''^) by (jSj. 



Using Galois coliomology, Schmoyer [12J computes the matrix of the Frobenius 
on the Tate module, up to a certain precision, if the self-pairings of the Tate 
pairing are degenerate. We use a similar approach and show that the precision 
up to which the Frobenius acts on the Tate module as a multiple of the identity 
is 2n — kij. Consequently, we recover information on the conductor of the endo- 
morphism ring of J by computing ki j. For m € Z, we will use a symplectic basis 
of J[£'"], i.e. a basis such that the matrix associated to the £™-Weil pairing is 



/ 

-10 



mod r. (7) 



Proposition 3. Let H be a hyperelliptic curve defined over a finite field ¥ q , and 
J its jacobian. Suppose that the Frobenius endomorphism tt is such that -k —1 is 
exactly divisible by £"■, for £ >3 prime. Then if Vi^End(j){'^) < 2n, we have 

ve,End(j) (tt) = 2n- k^j. (8) 
Proof. Let {Qi, Q2, 1, Q-2] a symplectic basis for the ^'^"-torsion (whose ma- 
trix is given by Equation (O) and let TT{Qg) = J2h=-2 a/i,gQ/!., with {ah,g)h,ge{-2,-i,i,2} 
in Z. By bilinearity, we have that 

2 

Te^{£''Q,,rQj) = We24Q,,n{Qj)-Qj)^W<,24Q,, ^ a,,,,Q,, - Q,) (9) 

h=-2 

2 

= VF,2„(Q„g,p--i n Wp^iQ^QhT-^-. (10) 

h=-2 

If j ^ ~i, we have that T^-n[i"-Q^^P^Qj) £ n^ke j. It follows that 

a_,j = (mod ^2"-fcf„7)^ 

for i ^ -j. If j = -i, then T^,. (^ Q,, t'Qj) = T4^^2„ (Qi, Qj)''^.^-i. Since the Tate 
pairing is /c^^j-antisymmetric we get 

a,,, EE a_,,_, (mod ^2"-*=). 

It remains to prove that Ui^i = ajj , for i,j £ {—2, —1, 1, 2}. Note that by Galois 
invariance, we have We2n.{n{Qi),TT{Qj)) — n{We2r,(Qi,Qj)) = Wpr^[Qi^Q.jY . 
For i = —j we have 

2 2 

W£2„(7r((5,,),7r(Q_j)) = W£2„( ^ a,,,,,Q,,, ^ Og-jQg) 

h=-2 3=-2 
2 2 2 

/i=-2g=-2 h=-2 
h^O g^O h^O,i 

2 2 2 

• n W,2„(a,,,Q„ag,_,Q<,) [] [| Wf2„ (Q„ g*)'^-''^*--' 

g=-2 s=-2 i=-2 



Since {Qi, Q2, Q-i, Q-2} is a symplectic basis and that ah,g = (mod ^"), for 
h 7^ —(7, then 

Since a^^i = a-i-i (mod ^2n-fe£,,/^^ follows that 

= (? for all i e {-2, -1, 1, 2}. 

Since at^i = 1 (mod ("'), it follows that m^i = b (mod £2n-/c{,j^^ j^j. some & S Z. 
By Lemmaini we have 2n — kij < vi^n). For the converse, let fc = 2n — Vf^Endj(7r) 
and -R, be two points in J[£"] such that Wi{R, S) = 1. It suffices to show that 
Tgn {R, S) is fc-degenerate. We write tt—1 — oi +a2a + 03/3 + 046', where 1, a, 13,9 
are a Z-basis of End(J). We take S such that S = £"S and we get 

T,„(i?, 5) = VK,.(i?,(^-l)(5)) = 

Since PF|(i?, 5*) = 1 and w^(gcd(a2, 03, 04)) = ^^n-fc^ have Ttr^{R,S) G ^^fc. 
Hence k > ki^j. This concludes the proof. 

Proposition [3] gives a method to compute to compute f^,Endj(7'') using pairings. 
Together with Lemma [3l this gives a criterion to check whether the endomor- 
phism ring of a jacobian is locally maximal at i. 

Theorem 2. Let H he a hyperelliptic curve defined over a finite field ¥q and J 
its jacobian. Suppose that the Frobenius endomorphism n is exactly divisible by 
n €: Z and that the conditions in Lemma\^ are satisfied. Then if v^_Qj^{'k) < 
2n, End{J) is a locally maximal order at £ if and only ifk^^j equals 2n— W£.e)^(7r). 

Proof. By Proposition [31 kg^j equals 2n — w^".o(7r), where O ~ End(J). By 
Lemma [31 the value of ^£".0^^ (tt) uniquely characterizes orders which are locally 
maximal at £. 

The following corollary reformulates the condition under which we may apply 
the criterion in Theorem [5] 

Corollary 1. Let H be a hyperelliptic curve defined over a finite field ¥q and 
J its jacobian. Let 7r = l + ai+a2(5 + 037 + 0477 be the decomposition of the 
Frobenius over a Z-basis ofOx- Thenk^^j > if and only if vi{gcd{a2, 03,04)) < 
2veXgcd{ai,a2,a3,a4)). 

We conclude this section by giving in Algorithm [T] a computational method 
which verifies whether the jacobian J of a genus 2 curve has locally maximal 
endomorphism ring. If fc^ j = 0, the algorithm aborts. By Lemma [H computing 
kg^j is equivalent to computing the greatest power of £ dividing all coefficients 
ttij, with i 7^ j of the matrix of the Frobenius on the Tate module. Equation [^l 
shows that in order to compute the £-adic valuation of these coefficients, it 
suffices to determine all the values Tin.[Qi, Qj), for i ^ —j. 



Algorithm 1 Checking whether the endomorphism ring is locally maximal 
INPUT: A jacobian J of a genus 2 curve defined over F, such that J[^"] C ^(F,), 

the Frobenius tt, a symplectic basis (Qi, (52, Q-i, Q-2) for J[£"] 
OUTPUT: The algorithm outputs true if End( J) is maximal at £ if ve,OK (^) < 2n. 



1 


for all i,j e {1,2, -1, -2} do 


2 


if i 7^ —j then 


3 


Compute tij ^ Ti,^{Qi, Qj) 


4 


else 


5 




6 


end if 


7 


end for 


8 


Let Count and check < 1. 


9 


while check 7^ Count do 


10 


check <— Count 


11 


for alli,j G {1,2,-1,-2} do 


12 


if tij 7^ 1 then 


13 




14 


Let Count — Count + 1 


15 


end if 


16 


end for 


17 


end while 


18 


ke,j ■'^ n — Count 


19 


if Count — then 


20 


abort 


21 


end if 


22 


if ki^j = 2n- ve^OxM then 


23 


return true 


24 


else 


25 


return false 


26 


end if 



5 Application to horizontal isogeny computation 

In this section, we are interested in computing horizontal isogenics, i.e. isogenics 
between Jacobians having the same endomorphism ring. Note that if J : Ji — J2 
is an isogeny such that Ji has maximal endomorphism ring at we distinguish 
two cases: either End(J2) is locally maximal at £, or End(J2) C End(Ji). In the 
last case we say that the isogeny is descending. 

Over the complex numbers, horizontal isogenics are given in terms of the ac- 
tion of the Shimura class group [13]. Let <Phe a. CM- type and let A be an abelian 
surface over C with complex multiplication by Ok, given by A = <C? /<P{I~^), 
where I is an ideal of Ok ■ The surface is principally polarized if there is a purely 
imaginary S Ok with Im(<?i(^)) > 0, for i S (1, 2}, and such that £,TIk = II 
(where Dk is the different {a € Ok ■ T^'^K/qioiOK) C Z}). Computing horizon- 
tal isogenies is usually done by using the action of the Shimura class group |13) . 
This group, that we denote by €{K), is defined as 

{(a, a) I a is a fractional O^f -ideal with aa = (a) with a G Kq totally positive}, 



where (a, a) ^ (b, /3) if and only if there exists u e K* with b = ua and /3 — uua. 
The action of (o, a) S <(L{K) on an principahy polarized abelian surface given by 
{!,(,) is given by the ideal (a/, af). This action is transitive and free [13' §14.6]. 

If the norm of a is coprime to the discriminant of Z[7r,7f]. the kernel of the 
horizontal isogeny corresponding to o is a subgroup of the ^-torsion invariate 
under the Frobenius endomorphism. Hence in order to compute the kernel, we 
need to compute the matrix of the Frobenius for some basis of the i'-torsion and 
then determine subspaces invariated by this matrix (see [31 Algorithm VI. 3. 4]). 
We show that, when a Jacobian with locally maximal order at £ is given, kernels 
of {£, ^)-horizontal isogenics are subgroups on which the Tate pairing is degen- 
erate. This result holds for any £ > 2 and is independent of the value of the 
discriminant of Z[7r,7f]. The resulting algorithm, whose complexity is analysed 
in Section [6l computes kernels of horizontal isogenics with only a few pairing 
computations. We state the following lemma for jacobians of genus 2 curves over 
finite fields, which are the framework for this paper. We note that the result 
holds for abelian varieties. 

Lemma 5. (a) Let Ji, J2 be jacobians of genus 2 curves defined over a finite 
field ¥q and / : Ji — >■ J2 an isogeny defined over ¥q which splits multiplication 
by d. Let A : Ji — > Ji 6e a principal polarization. Then for P G Ji{K), 
Q G Ji[m]{K) we have 

T^'{I{P),I{Q))^t:^{P,Q)^ 

where A/ : J2 J2 is the principal polarization such that IoXjoI = doX. 
(b) Let Ji, J2 be jacobians of genus 2 curves defined over ¥q and / : Ji — > J2 
an isogeny defined overWq which splits multiplication by m. Let P G Ji{K), 
Q G Ji[mm']{K) such that I{Q) is a m' -torsion point. 

T^',{I{P),im^T^ra'iP.Qy'', 

where A/ is a principal polarization of J2 such that loXjoI^moX. 
Proof, (a) It is easy to check that 5{I{P)) = I{S{P)). Hence for cr G Gk we have 

Wm(F/(P)(a),/(Q)) = VF„,(/(Fp(a)),/(Q)). 
By using [9_, Proposition 13. 2. b] 

W,^^'iIiFpia))jm = wt''°'iFp{a),Q). 

(b) The proof is immediate by using (a) and the fact that Tmm' {I (P) , I (Q)) = 
T^,{I{P)J{Q)). 

Lemma 6. Let H/¥q be a hyperelliptic curve and Di,D2 are two elements of 
J{¥q) of order n > 1. Let Di,D2 G Jni^q) such that £Di = Di and £D2 = 
Z?2. Then we have 



(a) IfDi,D2 e J{¥g), then 



(b) Suppose e>3.IfDi£ J{¥q)\JH{¥q), then 

Te^+iiDi,D2Y ^Tin{Di,D2). 

Proof. The proof is similar to to the one of |7j Lemma 4.6]. For completeness, 
we detail it in Appendice [HI 

Remark 1. Let G G W. By an argument similar to the one in Lemma[2l in order 
to determine the largest integer k such that T^n : G x G -> /i^fc is surjective, 
it suffices to determine the largest k such that all the self-pairings ™ (P, P) , 
with P G G, are primitive ^'^-th roots of unity. Let G and G' in W such that 
r-^G = r^'^G'. First note that P' S G' can be written as P' = P + L, with 
P e G and L e J[r-^]. Then by bilinearity 

(p', p') = (p, p)(r,,. (p, p)r,. (L, P))T,. (w^, PF) 

By Lemma [Hand given that L e J[^"~^], we have that Ti^{P',P') is a ^'^^ ^-th 
primitive root of unity if and only if T£n(P, P) is a primitive root of 

unity. This implies that in order to compute /c^.j it suffices to compute pairings 
over a set of representatives of W modulo the equivalence relation G ^ G' if and 
only if r-iG = r-iG'. 

We may now prove Theorem [T] 

Proof of Theorem 1. We assume that kj > 2. Otherwise, we use Lemma [5] and 
work over an extension field of F^. We denote by / : J — ?> J' the isogeny of kernel 
G. Suppose that G is such that the Tate pairing is non-degenerate over G x G. 
Then by applying Lemma [S] we have 

T£,.-i(/(Pi),/(P2)) e H),kt_j-i\fj.f,k,_j~2, 

for Pi , P2 e G. If J'[£"] is not defined over F^, then its endomorphism ring cannot 
be maximal at £, hence the isogeny is descending. Assume then that J'[£"] is 
defined over F,. Let A, -P2 S J'[i"] be such that /(Pi) = ^Pi, (-P2) = ^^2- Then 
TfTi(Pi,P2) S • We denote by G' — < Pi,f2 >• The subgroup 

G' may be chosen such that it is maximal isotropic with respect to the 
Weil pairing. It follows that kji > kj + 1. By Theorem [51 we deduce that 
the endomorphism ring of J' is not locally maximal at hence the isogeny is 
descending. 

Suppose now that the Tate pairing is degenerate over GxG. We distinguish two 
cases. 

Case 1. Suppose that J'[€"] is defined over Fg. With the same notations as above, 
we get that Ti^ (Pi, P2) £ • Let L C J'[£"] be a subgroup of rank 2 maximal 

isotropic with respect to the Weil pairing and consider Qi,Q2 & L\G' . Then 



r-igi,r-ig2 e Ker /t. since rf.-i(/t(gi),/t(g2)) e ^i^k,^J-^, it follows 
that Tin{Qi,Q2) G /i^fcj j-i. Hence fcj' < fc^^j. By Theorem[21 we conclude that 
the endomorphism ring of J' is locally maximal at £. 

Case 2. Suppose that J'[^"] is not defined over F,. Hence / is descending. We 
have 

r,„-i(/(Pi),/(P2))eA^,'=...-2. 

Let L C J'[£"~^] be a subgroup of rank 2 such that is maximal isotropic 

with respect to the Weil pairing and consider Qi,Q2 € L\G'. Then i"~^Qi, £^~'^Q2 G 
Ker /t. Since Tt^^l{I'' {Qi), {Q2)) € ^l^,>^,,J-^, it follows that Ti^-i{Qi,Q2) e 
lj,^kf j~3. Hence ,Endj' (''') = Wf,Endj(7'') which contradicts the hypothesis that 
/ is descending. 

In order to find all kernels of horizontal isogenics we search, among subgroups 
G G W (modulo the i?"^ ^-torsion), those for which the Tate pairing restricted 
to G X G maps to iJ.f>ki . If {gi, g2, Q-i, Q~2} is a symplectic basis for J[^"], 
then a subgroup of rank 2 generated by Aigi + A_ig_i + A2g2 + ^-2Q-2 and 

KQi + A'_ig_i + A^ga + x'_2Q-2, with a„a;- e f^, ij g {-2,-1,1,2}, is 

maximal isotropic with respect to the Weil pairing if the following equation is 
satisfied 

AiA'_i - A-iA'i + A2A'_2 - A_2A2 = 0. (11) 

Moreover, this subgroup has degenerate Tate pairing if the following equations 
are satisfied 

A,A,logT,„(g„g,) -Omodr^'^^+i (12) 

i,ie{i,2 -1.-2} 

J2 a,a; iogT,„(g„ g,) = mod r-'^-'+i (13) 

ije{i,2 -1 -2} 

J2 K^'j iogT,„(g„ g,) = mod r-'^^+i (i4) 

ijG{l,2 -1.-2} 

Example 1. We consider the jacobian of the hyperelliptic curve 
y2 ^ 5a,5 ^ 42;4 ^ 9g2;2 + 7a; + 2, 

defined over the finite field F127. The jacobian has maximal endomorphism ring 
at 5 and [EndJ : Z[7r,7f]] = 50. The ideal (5) decomposes as 5 = a^az in Ok- 
Hence there are two horizontal isogenies, which correspond to ideals ai and a^. 
under the Shimura class group action. The 5-torsion is defined over an extension 
field of degree 8 of the field F127, that we denote Fi27(t). Our computations with 
MAGMA found two subgroups of J[5], maximal isotropic with respect to the 
Weil pairing and with degenerate 5-Tate pairing. For lack of space, we give here 
the Mumford coordinates of the generators of one of these subgroups. 



+ {7Af + 25f + + llOt^ + 96i^ + 75f + 29t + 20)x 
+39f + 62<6 + 77t^ + A7t'^ + + &2f + 97i + 15, 
(116<^ + &lt^ + + 38*-^ + 7Qt^ + 109<2 + 62i + 7l)x + 98t^ 
+77i^ + + 7Qt^ + 81<3 + + 36i + 33) 
{x^ + (66<^ + 89<^ + 50<'^ + 12At^ + 91*^ + 102i2 + IQQt + 52)a; 
+119^^ + lAf + 126i^ + A2t^ + 42t^ + 85t^ + 12t + 77, 
(92t^ + + 94t5 + blt^ + 59t^ + 24t2 + 72t + ll)x 
+103^^ + 16^6 + 7t^ + lllf^ + 95t3 + 79t^ + Abt + 34) 

6 Complexity analysis 

In this section, we evaluate the complexity of Algorithm [T] and compare its 
performance to that of the Freeman-Lauter algorithm. Note that for a fixed 
i > 2, both algorithms perform computations in extension fields over which the 
£^-torsion, for a certain i"^ dividing [Ok ■ Z[7r,7f]], is rational. 

Checking locally maximal endomorphism rings. In Freeman and Lauter's algo- 
rithm, in order to check if End(J) is locally maximal at £, for > 2, it suf- 
fices to check that \/d and 77 are endomorphisms of J (see |4] Lemma 6]). If 
TT = ci + C2\fd (c3 -|- C4\/d)7El then we have 

2c2\/d = 7r + 7f-2ci (15) 
(4c2(c2 - c\d))j] = (2c2C3 - C4(7r + tt - 2ci))(7r - tt). (16) 

Moreover, Eisentrager and Lauter show that the index is \Ok ■ ^[tTiTt]] = 
2^02(03 — old), for some s E N. Hence, for a fixed £ > 2 dividing the index 
[Ok '■ 2[7r, 7f]], we need to consider an extension field over which J[i"] is defined, 
where u is the ^-adic valuation of the index. Meanwhile, Algorithm [T] performs 
computations over the smallest extension field containing the ^-torsion points. 
The degree of this extension field is smaller than £'^, by Proposition [TJ 

Notation. We denote by r the degree of the smallest extension field ¥qr such that 
the ^-torsion is Fgr -rational. 

We suppose that tt'' — 1 is exactly divisible by First, we need to compute 
a basis for the ^"-torsion. We assume that the zeta function of J/F^r and the 
factorization ^J{¥qr) = £^m are known in advance. In order to compute the 
generators of J[£"], we use Freeman and Lauter's probabilistic algorithm [S], 
which needs 0{rM{r)\ogq) operations in F^. We then compute a symplectic 



Note that we cannot always write vr in this form, but if this is not case, we can 
always replace tt by 2*7r, for some s € Z. 



basis of J[^"], by using an algorithm similar to Gram-Schmidt orthogonalization. 
In order to compute fcf^j, we use the values of the Tate pairing T^i(Qi,Qj) for 
i,j e {1, —1, 2, —2}. Computing the Tate pairing costs 0{M{r){n\og£ + r\ogq)) 
operations in F^, where the first term is the cost of Miller's algorithm and the 
second one is the cost for the final exponentiation. We conclude that the cost 
of Algorithm [T] is 0{M{r){r\ogq + nlogi)). The complexity of Freeman and 
Lauter's algorithm for endomorphism ring computation is dominated by the 
cost of computing the £-Sylow group of the Jacobian defined over the extension 
field containing the ^"-torsion, whose degree is r + (by Proposition [21) . The 
costs of the two algorithms are given in Table [TJ 

Table 1. Cost for checking locally maximal endomorphism rings at i 



Freeman and Lauter 


This work (Algorithm [1} 


0((r + e-'-)M{r + r-"-) log q) 


0(M (r) (r log g + n log £)) 



Computing horizontal isogenics. Both classical algorithms and our algorithm 
need to compute first a basis for the ^-torsion. As stated before, this costs 
0{rM{r)\ogq). The classical algorithm (see [U Algorithm VI. 3. 4]) computes 
subspaces which are invariant under the action of Frobenius. More precisely, 
this algorithm needs to compute the matrix of the Frobenius endomorphism 
(in 0{P) operations in F^r using a baby-step giant-step approach). We conclude 
that the overall complexity of this algorithm is 0{M {r){\og q + i"^)) . The method 
described in Section [5] computes a symplectic basis of the £" -torsion and solves a 
system of 4 homogenous equations of degree 2, with coefficients in F^. The cost 
of solving this system is polynomial in £ and thus negligible (€ is small). Our 
method for horizontal isogeny computation has the same cost as Algorithm [TJ 

7 Conclusion 

For an ordinary jacobian defined over a finite field, we have described a rela- 
tion between its endomorphism ring and some properties of the ^-Tate pairing. 
We deduced an efficient criterion for checking whether the jacobian is locally 
maximal at I and an algorithm computing kernels of {£, ^)-isogenies. 
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9 Appendix A 

We detail the proof of Lemma [HI 
Proof, (a) We can easily check that 



Note that these functions are F^-rational. By evaluating them at Di and raising 
to the power {q— 1)/£'^, we obtain the desired equahty. (b) Since div (/^n+i^Da) = 




div(/|„ wehaver<,.+i(Di,L»2) = r^„' (L»i,L»2), where rj^' isthe^-Tate 
pairing defined over F^c . We only need to show that 

T^y\Di,D2) = Te^{Di,D2) 

Note that we have 7r(Di) = Di+D^, where Dg is a point of order i. This implies 
that 

Di + 7r(5i) + 7r2(Di) + . . . + Tr^'^Di) ~ (Di ~ Di. 

Hence we get 

(¥ A - - (i + <j+-.- + <j^^^)(9-i) 

= h^MDi + T^{Di) + ... + Tr'-^{D,))^. 
By applying Weil's reciprocity law, we obtain 

where / is such that div(/) = + (7r(Z)i)) + . . . + (7r(Lii)) - eDi and that 
supp(/) n supp(£)2) = 0. Note that / is Fg-rational, so /(£)2)^"^ = 1- This 
concludes the proof. 



